Infinite Horizon Noncooperative Differential Games

For a non-cooperative differential game, the value functions of the various players satisfy a system of Hamilton-Jacobi equations. In the present paper, we consider a class of infinitehorizon games with nonlinear costs exponentially discounted in time. By the analysis of the value functions, we establish the existence of Nash equilibrium solutions in feedback form and provide results and counterexamples on their uniqueness and stability. 1 Introduction Problems of optimal control, or zero-sum differential games, have been the topic of an extensive literature. In both cases, an effective tool for the analysis of optimal solutions is provided by the value function, which satisfies a scalar Hamilton-Jacobi equation. Typically, this first order P.D.E. is highly non-linear and solutions may not be smooth. However, thanks to a very effective comparison principle, the existence and stability of solutions can be achieved in great generality by the theory of viscosity solutions, see [BC] and references therein. In comparison, much less is known about non-cooperative differential games. In a Nash equilibrium solution, the value functions for the various players now satisfy not a scalar but a system of Hamilton-Jacobi equations [F]. For this type of nonlinear systems, no general theorems on the existence or uniqueness of solutions are yet known. A major portion of the literature is concerned with games having linear dynamics and quadratic costs, see [WSE],[EW],[AFJ] and [PMC]. In this case, solutions are sought among quadratic functions. This approach effectively reduces the P.D.E. problem to a finite dimensional O.D.E.. However, it does not provide insight on the stability (or instability) of the solutions w.r.t. small non-linear perturbations. In [BS1] the first author studied a class of non-cooperative games with general terminal payoff, in one space dimension. Relying on recent advances in the theory of hyperbolic systems of conservation laws, some results on the existence and stability of Nash equilibrium solutions could be obtained. On the other hand, for games in several space dimensions and also in various onedimensional cases, the analysis in [BS2] shows that the corresponding H-J system is not hyperbolic, hence ill posed. In the present paper we begin exploring a class of non-cooperative differential games in infinite time horizon, with exponentially discounted costs. In one space dimension, the corresponding value functions satisfy a time-independent system of implicit O.D.E’s. Global solutions are sought within a class of absolutely continuous functions, imposing certain growth conditions as |x| → ∞, and suitable admissibility conditions at points where the gradient ux has a jump. 1 The dynamics of our system is very elementary, and the cost functions that we consider are small perturbations of linear ones. However, already in this simple setting we find cases where the problem has unique solution, and cases where infinitely many solutions exist. This provides a glimpse of the extreme complexity of the problem, for general non-cooperative N -player games with non-linear cost functions. The plan of the paper is as follows. In Section 2 we describe the differential game, introducing the basic notations and definitions. In Section 3 we prove that, from an admissible solution to the O.D.E. for the value function, one can always recover a Nash equilibrium solution to the differential game. The relevance of our admissibility conditions is then highlighted by two examples. The existence and uniqueness of global admissible solutions to the H-J system for the value functions is then studied in Sections 3 and 4. We first consider the cooperative case, where both players wish to move the state of the system in the same direction. In the case with terminal payoff, this situation was leading to a well-posed hyperbolic Cauchy problem [BS1]. As expected, in the infinite-horizon case we still obtain an existence and uniqueness result. Subsequently, we consider the case of conflicting interests, where the players wish to steer the system in opposite directions. In the case with terminal payoff, this situation leads to an ill-posed Cauchy problem, as shown in [BS2]. Somewhat surprisingly, we find that the corresponding infinite-horizon case can have unique or multiple solutions, depending on the values of certain parameters. 2 Basic definitions Consider an m-persons non-cooperative differential game, with dynamics ẋ = m ∑ i=1 fi(x, αi), αi(t) ∈ Ai , x ∈ IR. (2.1) Here t 7→ αi(t) is the control chosen by the i-th player, within a set of admissible control values Ai ⊆ IR. We will study the discounted, infinite horizon problem, where the game takes place on an infinite interval of time [0, ∞[ , and each player has only a running cost, discounted exponentially in time. More precisely, for a given initial data x(0) = y ∈ IR , (2.2) the goal of the i-th player is to minimize the functional Ji(α) . = ∫ ∞ 0 e−t ψi ( x(t), αi(t) ) dt , (2.3) where t 7→ x(t) is the trajectory of (2.1). By definition, an m-tuple of feedback strategies αi = αi (x), i = 1, . . . ,m, represents a Nash non-cooperative equilibrium solution for the differential game (2.1)-(2.2) if the following holds. For every i ∈ {1, . . . ,m}, the feedback control αi = α∗(x) provides a solution to the the optimal control problem for the i-th player, min α(·) Ji(α) , (2.4) where the dynamics of the system is ẋ = fi(x, αi) + ∑ j 6=i fj(x, α ∗ j (x)), αi(t) ∈ Ai . (2.5) 2 More precisely, we require that, for every initial data y ∈ IR, the Cauchy problem ẋ = m ∑ j=1 fj ( x, αj (x) ) , x(0) = y , (2.6) should have at least one Caratheodory solution t 7→ x(t), defined for all t ∈ [0,∞[ . Moreover, for every such solution and each i = 1, . . . ,m, the cost to the i-th player should provide the minimum for the optimal control problem (2.4)-(2.5). We recall that a Caratheodory solution is an absolutely continuous function t 7→ x(t) which satisfies the differential equation in (2.6) at almost every t > 0. Nash equilibrium solutions in feedback form can be obtained by studying a related system of P.D.E’s. Assume that a value function u(y) = (u1, . . . , un)(y) exists, so that ui(y) represents the cost for the i-th player when the initial state of the system is x(0) = y and the strategies α1, . . . , α ∗ m are implemented. By the theory of optimal control, see for example [BC], on regions where u is smooth, each component ui should provide a solution to the corresponding scalar Hamilton-JacobiBellman equation. The vector function u thus satisfies the stationary system of equations ui(x) = Hi(x, ∇u1, . . . ,∇um), (2.7) where the Hamiltonian functions Hi are defined as follows. For each pj ∈ IR, assume that there exists an optimal control value αj (x, pj) such that pj · fj ( x, αj (x, pj) ) + ψj ( x, αj (x, pj) ) = min a∈Aj { pj · fj(x, a) + ψj(x, a) } . (2.8) Then Hi(x, p1, . . . , pm) . = pi · m ∑ j=1 fj ( x, αj (x, pj) ) + ψi ( x, αi (x, pi) ) . (2.9) A rich literature is currently available on optimal control problems and on viscosity solutions to the corresponding scalar H-J equations. However, little is yet known about non-cooperative differential games, apart from the linear-quadratic case. In this paper we begin a study of this class of differential games, with two players in one space dimension. Our main interest is in the existence, uniqueness and stability of Nash equilibrium solutions in feedback form. When x is a scalar variable, (2.7) reduces to a system of implicit O.D.E’s: ui = Hi(x, u ′ 1, . . . , u ′ m) . (2.10) In general, this system will have infinitely many solutions. To single out a (hopefully unique) admissible solution, corresponding to a Nash equilibrium for the differential game, additional requirements must be imposed. These are of two types: (i) Asymptotic growth conditions as |x| → ∞. (ii) Jump conditions, at points where the derivative u′ is discontinuous. To fix the ideas, consider a game with the simple dynamics ẋ(t) = α1(t) + · · · + αm(t) , (2.11) and with cost functionals of the form Ji(α) . = ∫ ∞ 0 e−t [ hi ( x(t)) + ki ( x(t) ) αi (t) 2 ]